In electronic document review, such as when reviewing documents for legal cases, accuracy is vital to determining whether an electronic document is relevant to a particular case or topic. Traditionally, human reviewers review and code these electronic documents to decide their relevance. Unfortunately, analysis of documents by human reviewers may be inaccurate for many reasons. For example, a single reviewer may be inconsistent in reviewing multiple documents. In addition, manual coding takes valuable time and resources and may not be cost-effective for large volumes of electronic documents.
Furthermore, documents are often allocated to multiple reviewers for faster review. However, reviewers may not be consistent with each other, and each reviewer may have a different level of accuracy or speed. In some cases, machine learning methods may be used to train a predictive model to automatically review electronic documents for relevancy. While this may increase the overall consistency of reviews and decrease costs, the accuracy of a model trained by machine learning is dependent on the accuracy of the training data, which may be manually coded and flawed. Accordingly, the instant disclosure identifies and addresses a need for additional and improved systems and methods for automated document review and quality control.